Skip to content

huowang#1021

Closed
arthas-171 wants to merge 1 commit intoapache:trunkfrom
arthas-171:trunk
Closed

huowang#1021
arthas-171 wants to merge 1 commit intoapache:trunkfrom
arthas-171:trunk

Conversation

@arthas-171
Copy link
Copy Markdown

@arthas-171 arthas-171 commented Jun 27, 2019

I encountered a problem when I converted edits files from XML to binary using the HDFS oev command
Usually this conversion operation will not be a problem, but my XML culture contains an OP_TRUNCATE operation node, in this case, the error is reported, the specific error is as follows. (sorry, I can only take a screenshot because the company environment does not allow copying code, the name of the file is not NameNode's default name because I backed up the edits file for testing).
image

The contents of the XML file are as follows
image
I found the error-reporting code and found that the blockFromXml method passed the entire data node instead of the subordinate BLOCK node. I think this was the reason. So I modified the code and recompiled it. Then I installed a single node Hadoop cluster and tested the HDFS oev command conversion file successfully. This is my first time to submit a patch. My English is not particularly good, I am not sure. Should the modified code be submitted in this way?

I hope I can contribute to the community.

@hadoop-yetus
Copy link
Copy Markdown

hadoop-yetus commented Jun 27, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 65 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1080 trunk passed
+1 compile 63 trunk passed
+1 checkstyle 48 trunk passed
+1 mvnsite 66 trunk passed
+1 shadedclient 780 branch has no errors when building and testing our client artifacts.
+1 javadoc 47 trunk passed
0 spotbugs 171 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 168 trunk passed
_ Patch Compile Tests _
+1 mvninstall 60 the patch passed
+1 compile 54 the patch passed
+1 javac 54 the patch passed
-0 checkstyle 42 hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 327 unchanged - 0 fixed = 328 total (was 327)
+1 mvnsite 61 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 719 patch has no errors when building and testing our client artifacts.
+1 javadoc 48 the patch passed
+1 findbugs 168 the patch passed
_ Other Tests _
-1 unit 6230 hadoop-hdfs in the patch failed.
+1 asflicense 36 The patch does not generate ASF License warnings.
9794
Reason Tests
Failed junit tests hadoop.hdfs.TestDFSStripedOutputStream
hadoop.hdfs.server.namenode.TestNameNodeMXBean
hadoop.hdfs.server.diskbalancer.TestDiskBalancer
hadoop.hdfs.TestDFSStripedInputStream
hadoop.hdfs.web.TestWebHdfsTimeouts
hadoop.hdfs.server.datanode.TestDirectoryScanner
hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
hadoop.hdfs.server.namenode.TestFSImage
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/1/artifact/out/Dockerfile
GITHUB PR #1021
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux b89a37b2e035 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / be80334
Default Java 1.8.0_212
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/1/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/1/testReport/
Max. process+thread count 3962 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/1/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@arthas-171
Copy link
Copy Markdown
Author

arthas-171 commented Jul 1, 2019

Can I describe how to reproduce bugs and test modified patches?
1 TRUNCATE operations are performed on any file in hdfs, such as Hadoop fs-truncate-w 100 /test.txt,
2 Use the HDFS oev command to parse the edits file generated by the first step from binary format to XML format, such as HDFS oev -i edits -o edits.xml
3 Then reverse serialization of the file from XML to binary based on the results of the second step, such as HDFS oev-i edits.xml-o edits.binary-p binary
4 This bug will appear, and I also submitted the report HDFS-14604 on jira.
5 I fixed the bug in the code I submitted this time, compiled this version of the code and then decompressed it in Linux. Using the oev command provided by this version directly, the operation described in the third step can be successfully executed.

@hadoop-yetus
Copy link
Copy Markdown

hadoop-yetus commented Jul 19, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 61 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 1 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1044 trunk passed
+1 compile 59 trunk passed
+1 checkstyle 43 trunk passed
+1 mvnsite 61 trunk passed
+1 shadedclient 703 branch has no errors when building and testing our client artifacts.
+1 javadoc 49 trunk passed
0 spotbugs 157 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 154 trunk passed
_ Patch Compile Tests _
+1 mvninstall 61 the patch passed
+1 compile 58 the patch passed
+1 javac 58 the patch passed
-0 checkstyle 40 hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 327 unchanged - 0 fixed = 328 total (was 327)
+1 mvnsite 65 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 718 patch has no errors when building and testing our client artifacts.
+1 javadoc 45 the patch passed
+1 findbugs 169 the patch passed
_ Other Tests _
-1 unit 6865 hadoop-hdfs in the patch failed.
+1 asflicense 31 The patch does not generate ASF License warnings.
10294
Reason Tests
Failed junit tests hadoop.hdfs.tools.TestDFSAdminWithHA
hadoop.hdfs.TestFileCreation
hadoop.hdfs.web.TestWebHdfsTimeouts
hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks
hadoop.hdfs.server.datanode.TestDirectoryScanner
hadoop.hdfs.tools.TestECAdmin
hadoop.hdfs.TestInjectionForSimulatedStorage
Subsystem Report/Notes
Docker Client=18.09.8 Server=18.09.8 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/2/artifact/out/Dockerfile
GITHUB PR #1021
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux f00c45f69dcd 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 4e66cb9
Default Java 1.8.0_212
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/2/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/2/testReport/
Max. process+thread count 3886 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/2/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link
Copy Markdown

hadoop-yetus commented Jul 26, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 93 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1090 trunk passed
+1 compile 60 trunk passed
+1 checkstyle 42 trunk passed
+1 mvnsite 60 trunk passed
+1 shadedclient 716 branch has no errors when building and testing our client artifacts.
+1 javadoc 47 trunk passed
0 spotbugs 166 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 165 trunk passed
_ Patch Compile Tests _
+1 mvninstall 63 the patch passed
+1 compile 54 the patch passed
+1 javac 54 the patch passed
-0 checkstyle 44 hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 327 unchanged - 0 fixed = 328 total (was 327)
+1 mvnsite 77 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 733 patch has no errors when building and testing our client artifacts.
+1 javadoc 46 the patch passed
+1 findbugs 158 the patch passed
_ Other Tests _
-1 unit 6590 hadoop-hdfs in the patch failed.
-1 asflicense 32 The patch generated 3 ASF License warnings.
10131
Reason Tests
Failed junit tests hadoop.hdfs.TestLeaseRecovery
hadoop.hdfs.TestRead
hadoop.hdfs.TestMaintenanceState
hadoop.hdfs.server.blockmanagement.TestHostFileManager
hadoop.hdfs.server.blockmanagement.TestPendingReconstruction
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/3/artifact/out/Dockerfile
GITHUB PR #1021
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux a118eca0d4c5 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / c0a0c35
Default Java 1.8.0_212
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/3/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/3/testReport/
asflicense https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/3/artifact/out/patch-asflicense-problems.txt
Max. process+thread count 4693 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/3/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@jojochuang
Copy link
Copy Markdown
Contributor

jojochuang commented Jul 26, 2019

Hi, thanks for the contribution, please follow the How to Contribute wiki https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute#HowToContribute-CreatingaGitHubpullrequest
Specifically, file a ticket at Apache Hadoop JIRA, and prepend your git commit message with the JIRA id.

@arthas-171
Copy link
Copy Markdown
Author

arthas-171 commented Jul 29, 2019 via email

@hadoop-yetus
Copy link
Copy Markdown

hadoop-yetus commented Aug 2, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 78 Docker mode activated.
_ Prechecks _
+1 dupname 1 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1223 trunk passed
+1 compile 70 trunk passed
+1 checkstyle 51 trunk passed
+1 mvnsite 74 trunk passed
+1 shadedclient 903 branch has no errors when building and testing our client artifacts.
+1 javadoc 57 trunk passed
0 spotbugs 191 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 189 trunk passed
_ Patch Compile Tests _
+1 mvninstall 72 the patch passed
-1 compile 18 hadoop-hdfs in the patch failed.
-1 javac 18 hadoop-hdfs in the patch failed.
-0 checkstyle 45 hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 327 unchanged - 0 fixed = 328 total (was 327)
+1 mvnsite 71 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 824 patch has no errors when building and testing our client artifacts.
+1 javadoc 51 the patch passed
+1 findbugs 189 the patch passed
_ Other Tests _
-1 unit 6324 hadoop-hdfs in the patch failed.
-1 asflicense 35 The patch generated 1 ASF License warnings.
10331
Reason Tests
Failed junit tests hadoop.hdfs.server.namenode.TestReencryption
hadoop.hdfs.TestDFSClientRetries
hadoop.hdfs.server.datanode.TestLargeBlockReport
hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/4/artifact/out/Dockerfile
GITHUB PR #1021
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 7dfdbd3029a0 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 17e8cf5
Default Java 1.8.0_212
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/4/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/4/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/4/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/4/testReport/
asflicense https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/4/artifact/out/patch-asflicense-problems.txt
Max. process+thread count 2732 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/4/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link
Copy Markdown

hadoop-yetus commented Aug 8, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 53 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1362 trunk passed
+1 compile 66 trunk passed
+1 checkstyle 44 trunk passed
+1 mvnsite 76 trunk passed
+1 shadedclient 862 branch has no errors when building and testing our client artifacts.
+1 javadoc 54 trunk passed
0 spotbugs 177 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 174 trunk passed
_ Patch Compile Tests _
+1 mvninstall 65 the patch passed
+1 compile 59 the patch passed
+1 javac 59 the patch passed
-0 checkstyle 47 hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 327 unchanged - 0 fixed = 328 total (was 327)
+1 mvnsite 75 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 824 patch has no errors when building and testing our client artifacts.
+1 javadoc 50 the patch passed
+1 findbugs 186 the patch passed
_ Other Tests _
-1 unit 5749 hadoop-hdfs in the patch failed.
-1 asflicense 55 The patch generated 5 ASF License warnings.
9860
Reason Tests
Failed junit tests hadoop.hdfs.TestReadStripedFileWithDecoding
hadoop.hdfs.TestEncryptionZonesWithKMS
hadoop.hdfs.TestReplaceDatanodeFailureReplication
hadoop.hdfs.server.datanode.TestLargeBlockReport
hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy
hadoop.hdfs.TestMaintenanceState
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/5/artifact/out/Dockerfile
GITHUB PR #1021
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 65dbb8d5cd4a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 70b4617
Default Java 1.8.0_212
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/5/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/5/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/5/testReport/
asflicense https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/5/artifact/out/patch-asflicense-problems.txt
Max. process+thread count 4925 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/5/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link
Copy Markdown

hadoop-yetus commented Aug 8, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 99 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1283 trunk passed
+1 compile 69 trunk passed
+1 checkstyle 47 trunk passed
+1 mvnsite 63 trunk passed
+1 shadedclient 907 branch has no errors when building and testing our client artifacts.
+1 javadoc 60 trunk passed
0 spotbugs 184 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 182 trunk passed
_ Patch Compile Tests _
+1 mvninstall 73 the patch passed
+1 compile 65 the patch passed
+1 javac 65 the patch passed
-0 checkstyle 50 hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 327 unchanged - 0 fixed = 328 total (was 327)
+1 mvnsite 79 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 824 patch has no errors when building and testing our client artifacts.
+1 javadoc 55 the patch passed
+1 findbugs 190 the patch passed
_ Other Tests _
-1 unit 6256 hadoop-hdfs in the patch failed.
+1 asflicense 37 The patch does not generate ASF License warnings.
10417
Reason Tests
Failed junit tests hadoop.hdfs.server.datanode.TestLargeBlockReport
hadoop.hdfs.TestMultipleNNPortQOP
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/6/artifact/out/Dockerfile
GITHUB PR #1021
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 567d4773bb05 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 397a563
Default Java 1.8.0_212
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/6/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/6/testReport/
Max. process+thread count 3228 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/6/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link
Copy Markdown

hadoop-yetus commented Aug 16, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 40 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1037 trunk passed
+1 compile 56 trunk passed
+1 checkstyle 43 trunk passed
+1 mvnsite 65 trunk passed
+1 shadedclient 723 branch has no errors when building and testing our client artifacts.
+1 javadoc 45 trunk passed
0 spotbugs 157 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 154 trunk passed
_ Patch Compile Tests _
+1 mvninstall 58 the patch passed
+1 compile 49 the patch passed
+1 javac 49 the patch passed
-0 checkstyle 39 hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 327 unchanged - 0 fixed = 328 total (was 327)
+1 mvnsite 56 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 686 patch has no errors when building and testing our client artifacts.
+1 javadoc 47 the patch passed
+1 findbugs 158 the patch passed
_ Other Tests _
-1 unit 4948 hadoop-hdfs in the patch failed.
+1 asflicense 28 The patch does not generate ASF License warnings.
8280
Reason Tests
Failed junit tests hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
hadoop.hdfs.server.datanode.TestDirectoryScanner
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/7/artifact/out/Dockerfile
GITHUB PR #1021
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 9c9aed7bc9a0 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / e356e4f
Default Java 1.8.0_212
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/7/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/7/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/7/testReport/
Max. process+thread count 4037 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/7/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link
Copy Markdown

hadoop-yetus commented Aug 20, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 62 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1270 trunk passed
+1 compile 65 trunk passed
+1 checkstyle 46 trunk passed
+1 mvnsite 69 trunk passed
+1 shadedclient 785 branch has no errors when building and testing our client artifacts.
+1 javadoc 55 trunk passed
0 spotbugs 177 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 175 trunk passed
_ Patch Compile Tests _
+1 mvninstall 65 the patch passed
+1 compile 57 the patch passed
+1 javac 57 the patch passed
-0 checkstyle 44 hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 327 unchanged - 0 fixed = 328 total (was 327)
+1 mvnsite 64 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 743 patch has no errors when building and testing our client artifacts.
+1 javadoc 49 the patch passed
+1 findbugs 202 the patch passed
_ Other Tests _
-1 unit 7409 hadoop-hdfs in the patch failed.
-1 asflicense 32 The patch generated 3 ASF License warnings.
11258
Reason Tests
Failed junit tests hadoop.hdfs.server.namenode.TestStorageRestore
hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand
hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
hadoop.hdfs.server.datanode.TestDirectoryScanner
hadoop.hdfs.server.namenode.TestFsck
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/8/artifact/out/Dockerfile
GITHUB PR #1021
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 3f055a58faad 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 094d736
Default Java 1.8.0_222
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/8/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/8/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/8/testReport/
asflicense https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/8/artifact/out/patch-asflicense-problems.txt
Max. process+thread count 3889 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/8/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link
Copy Markdown

hadoop-yetus commented Aug 22, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 36 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1325 trunk passed
+1 compile 65 trunk passed
+1 checkstyle 53 trunk passed
+1 mvnsite 78 trunk passed
+1 shadedclient 860 branch has no errors when building and testing our client artifacts.
+1 javadoc 52 trunk passed
0 spotbugs 172 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 170 trunk passed
_ Patch Compile Tests _
+1 mvninstall 63 the patch passed
+1 compile 57 the patch passed
+1 javac 57 the patch passed
-0 checkstyle 45 hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 327 unchanged - 0 fixed = 328 total (was 327)
+1 mvnsite 63 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 801 patch has no errors when building and testing our client artifacts.
+1 javadoc 56 the patch passed
+1 findbugs 197 the patch passed
_ Other Tests _
-1 unit 5793 hadoop-hdfs in the patch failed.
+1 asflicense 49 The patch does not generate ASF License warnings.
9811
Reason Tests
Failed junit tests hadoop.hdfs.TestMultipleNNPortQOP
hadoop.hdfs.TestLeaseRecovery2
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/9/artifact/out/Dockerfile
GITHUB PR #1021
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux db9b81253a3f 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 69ddb36
Default Java 1.8.0_222
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/9/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/9/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/9/testReport/
Max. process+thread count 2921 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/9/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link
Copy Markdown

hadoop-yetus commented Aug 27, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 42 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1148 trunk passed
+1 compile 68 trunk passed
+1 checkstyle 52 trunk passed
+1 mvnsite 63 trunk passed
+1 shadedclient 815 branch has no errors when building and testing our client artifacts.
+1 javadoc 54 trunk passed
0 spotbugs 163 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 160 trunk passed
_ Patch Compile Tests _
+1 mvninstall 63 the patch passed
+1 compile 57 the patch passed
+1 javac 57 the patch passed
-0 checkstyle 46 hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 327 unchanged - 0 fixed = 328 total (was 327)
+1 mvnsite 63 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 746 patch has no errors when building and testing our client artifacts.
+1 javadoc 51 the patch passed
+1 findbugs 172 the patch passed
_ Other Tests _
-1 unit 5220 hadoop-hdfs in the patch failed.
+1 asflicense 40 The patch does not generate ASF License warnings.
8925
Reason Tests
Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
hadoop.hdfs.TestLargeBlock
hadoop.hdfs.TestDFSClientRetries
hadoop.hdfs.TestLeaseRecovery2
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/10/artifact/out/Dockerfile
GITHUB PR #1021
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux d2c8e9f8e56f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 3329257
Default Java 1.8.0_222
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/10/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/10/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/10/testReport/
Max. process+thread count 3884 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/10/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link
Copy Markdown

hadoop-yetus commented Sep 3, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 39 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1097 trunk passed
+1 compile 64 trunk passed
+1 checkstyle 54 trunk passed
+1 mvnsite 76 trunk passed
+1 shadedclient 834 branch has no errors when building and testing our client artifacts.
+1 javadoc 60 trunk passed
0 spotbugs 191 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 188 trunk passed
_ Patch Compile Tests _
+1 mvninstall 74 the patch passed
+1 compile 60 the patch passed
+1 javac 60 the patch passed
-0 checkstyle 54 hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 327 unchanged - 0 fixed = 328 total (was 327)
+1 mvnsite 74 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 765 patch has no errors when building and testing our client artifacts.
+1 javadoc 57 the patch passed
+1 findbugs 171 the patch passed
_ Other Tests _
-1 unit 5333 hadoop-hdfs in the patch failed.
-1 asflicense 38 The patch generated 3 ASF License warnings.
9098
Reason Tests
Failed junit tests hadoop.hdfs.TestDecommission
hadoop.hdfs.TestPread
hadoop.hdfs.TestErasureCodingPoliciesWithRandomECPolicy
hadoop.hdfs.TestHDFSFileSystemContract
hadoop.hdfs.TestErasureCodeBenchmarkThroughput
hadoop.hdfs.tools.TestDFSAdminWithHA
hadoop.hdfs.TestClientReportBadBlock
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/11/artifact/out/Dockerfile
GITHUB PR #1021
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 64b314495650 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 915cbc9
Default Java 1.8.0_222
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/11/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/11/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/11/testReport/
asflicense https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/11/artifact/out/patch-asflicense-problems.txt
Max. process+thread count 4218 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1021/11/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@aajisaka
Copy link
Copy Markdown
Member

aajisaka commented Sep 11, 2019

Hi @arthas-171
Would you create an issue in ASF jira before submitting a pull request?
This is a guide for contributors: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute

@arthas-171
Copy link
Copy Markdown
Author

arthas-171 commented Sep 11, 2019

hi @aajisaka
yes i have created a issue ,HDFS-14604 https://issues.apache.org/jira/browse/HDFS-14604

shanthoosh pushed a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
@rmatharu @prateekm
SAMZA-2213: Adding writing of container.metadata file, and moving exc…
66f29e8 
…eption writing logic to DiagnosticsManager

This PR
* Removes ListGauge from everywhere.
* Adds logic to LocalContainerRunner to write metadata, that is useful when combined with java's -XX:OnOutOfMemoryError=string option.

* Adds DiagnosticsManager to publish error info to diagnostics stream instead of metrics reporter.
* Adds logic to SamzaContainer to write the DiagnosticsManager while re-using the SystemProducer from the MetricsReporter.

* Cleansup ContainerProcessManagerMetrics.

* Adds DiagnosticsManager to ContainerProcessManager and the ClusterBasedJC

Author: Ray Matharu <rmatharu@linkedin.com>
Author: rmatharu <40646191+rmatharu@users.noreply.github.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes apache#1021 from rmatharu/job-metadata
@MarcelRaschke MarcelRaschke mentioned this pull request Aug 19, 2022
Merged
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 22, 2025

We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.

@github-actions github-actions bot added the Stale label Dec 22, 2025
@github-actions github-actions bot closed this Dec 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Stale

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@arthas-171 @hadoop-yetus @jojochuang @aajisaka